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ABSTRACT 

Information and communication technology (ICT) is one of the most important driving forces promoting 
economic growth in the economy. Flowever, one puzzling question concerns the efficient and effective impact of 
ICT on educational outputs and outcomes. Therefore, the purpose of the paper is to discuss and review some 
previous research studies on development of the information society and its impact on educational 
outputs/outcomes at regional level. Respectively, a definition, measurements and the empirical application of the 
efficiency and effectiveness of the ICT at the regional (NUTS 2) level in the EU is considered (based on 2007- 
2011 average data). The research findings suggest that a wide range of NUTS 2 regions is characterized by a 
relatively low efficiency rate of transforming information society progress into educational outputs/outcomes, 
particularly in Eastern and Southern Europe. 
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INTRODUCTION 

Widespread use of the information and communication technology (ICT) and also the Internet and the Word 
Wide Web (WWW) have led to the development of what is often referred to as the information society. One 
puzzling question concerns the effective impact of information society development on educational outputs and 
outcomes. As ICTs are being increasingly used in education, indicators to monitor their impact and demonstrate 
accountability to funding sources and the public are ever more needed. Indicators are required to show the 
relationships between technology use and educational performance. There is also a need to show that education 
should be seen as using technology not only as an end in itself, but as a means to promote creativity, 
empowerment and equality and produce efficient learners and problem solvers. Many academic researchers have 
tried to answer this question at theoretical and empirical levels. They have faced two main difficulties. On one 
hand, student performance is hard to observe and there is still confusion about its definition. On the other, ICT 
entails evolving technologies and their effects are difficult to isolate from their environment. Consequently, the 
relationship between the development of the information society and educational performance is unclear, and 
contradictory results are presented in the literature (Youssef & Dahmani, 2008). Recently, there have been 
several initiatives to assess and monitor the efficiency of ICT sector indicators and their impact on education. 
Some studies conclude that the information society’s development has a positive impact on education outcomes 
(see Balanskat et al., 2006; Yusuf, & Afolabi, 2010; Shaikh, 2009; Jayson, 2008; Shaheeda et al., 2007, Iqbal & 
Ahmed, 2010; Hameed, 2006; Amjad, 2006; Khan, and Shah, 2004; Aristovnik, 2012). Conversely, others point 
out that the impact is unclear (see Trucano, 2005; Cox, and Marshall, 2007; Machin et al., 2006; Leuven et al., 
2004). Nevertheless, the existing literature reveals a gap in the empirical knowledge of the information society 
and its impact and efficiency on educational outputs/outcomes at the EU regional level. Therefore, this paper 
seeks to fill this gap. 

The paper is structured as follows: first, a brief review of the methodology and data is presented and the 
specifications of the models are defined. The next section outlines the results of a non-parametric efficiency 
analysis in order to assess the impact of information society development on educational performance. The final 
section provides some concluding remarks. 

METHODOLOGY AND DATA 

We adopted the mathematical development of Data Envelopment Analysis (DEA) by Chames et al. (1978) who 
built on the work of Farrell (1957) and others. DEA is a linear programming-based methodology that has proven 
to be a successful tool for measuring efficiency. It computes the comparative ratio of outputs to inputs for each 
unit, with the score expressed as 0-100%. It is used to identify best practices and is increasingly becoming a 
popular and practical management tool. DEA was initially used to investigate the relative efficiency of non¬ 
profit organizations but now its use has spread to hospitals, schools, banks and network industries, among others 
(Avkiran, 2001). DEA empirically identifies the best producers by forming the efficient frontier (composed by 
efficient producers) based on observed indicators from all producers. We refer to the producers as decision¬ 
making units (DMUs). A DMU with a score of less than 100% is inefficient compared to other units. 
Consequently, DEA bases the resulting efficiency scores and potential efficiency improvements entirely on the 
actual performance of other DMUs, free of any questionable assumptions regarding the mathematical form of the 
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underlying production function. We use the DEA methodology to evaluate the relative efficiency of each region 
as it converts, for instance, Internet use into tertiary educational attainment. In our analysis we focus on the 
NUTS classification (Nomenclature of territorial units for statistics), which is a hierarchical system for dividing 
up the economic territory of the EU for the statistical, regional and other purposes. Indeed, we identify the 
regions (NUTS 2) as the DMUs. The Level 2 of the nomenclature (NUTS 2) has 271 regions in the considered 
period. Let n (=271) be the number of (EU NUTS 2) regions in the data set. Let Xy be the amount of input i 
consumed by Region j, for i = 1 and j = 1,2, ...,271. Let Yj be the number of patent applications by Region j, for 
j = 1,2, ..., 271. We are now ready to present the output-oriented DEA model for Region k, k = 1, 2, ..., 271. 
We must solve one such linear programming model for each region. Mathematically, the technical efficiency of 
each DMU is computed as: 


subject to 


Max <j> k 


271 

I jXy ^X ik for i = 1,2,3 
7=1 

271 

Z *jYj>faY k 
7=1 

271 

X Aj= 1 
7=1 

Xj > 0 for j = 1,2,...,271 
</k> 0 


(1) 

( 2 ) 

(3) 

(4) 

(5) 

(6) 


We observe that setting X k =l, 2j=0 for jfk and 0 k =l is a feasible but not necessarily optimal solution to the linear 
program for Region k. This implies that 0 k *, the optimal value of 0 k , must be greater than or equal to 1. The 
optimal value, 0 k *, is the overall inverse efficiency of DMU k, which represents one plus the proportion by 
which Region k can increase its patent applications. For instance, if 0 k *=l.lO, then Region k can increase its 
output by 10% without increasing any of its inputs. We refer to E k *=l/0 k * as the overall efficiency of region k. 
Thus, if 0 k *=l.lO then E k *=0.91 and we can say that Region k is 91% efficient overall. The left-hand side of 
Equations (2) and (3) are weighted averages because of Equations (4) and (5), of the inputs and output, 
respectively, of the 271 regions. At optimality, that is with the Xj replaced by Xj *, we call the left-hand side of 
Equations (2) and (3) the target inputs and target outputs respectively, for Region k. 

In the majority of studies using DEA the data are analysed cross-sectionally, with each decision-making unit 
(DMU) - in our case a region - being observed only once. Nevertheless, data on DMUs are often available over 
multiple time periods. In such cases, it is possible to perform DEA over time where each DMU in each time 
period is treated as if it were a distinct DMU. However, in our case the data set for all the tests in the study 
includes average (available) data for the 2006-2009 period (for inputs) and for the 2010-2011 period (for 
outputs) in order to evaluate long-term efficiency measures as the effects of information society are characterized 
by time lags in selected EU (NUTS 2) regions. The inputs utilized are households that have Internet access at 
home (% of households with at least one member aged 16 to 74) ( HIA) and the percentage of households with 
broadband access in relation to households with Internet access (% of households with at least one member aged 
16 to 74 and Internet access) (HBA). As the total efficiency score also comprises contributions from the non¬ 
discretionary (i.e. uncontrolled) variables, we included selected non-discretionary variables. In our case, regional 
gross domestic product (PPS per inhabitant in % of the EU-27 average) (GDP) and population density 
(inhabitants per km 2 ) (POPDEN) were included in the model. In the analysis, the output/outcome can be in the 
form of pupils and students in upper secondary and post-secondary non-tertiary education (ISCED 3-4) (% of the 
population aged 15-24 years) ( SECED ), students in tertiary education (ISCED 5-6) (% of the population aged 
20-24 years) (TERED), pupils and students at all levels of education (ISCED 0-6) (% of total population) 
(EDTOT ), tertiary educational attainment (% of the population aged 25-64) ( TERAT) and employment rate (of 
the age group 15-64, in %) {EMPLOY). Based on data availability, up to 146 (out of 271+2 (from Croatia)) EU 
(NUTS 2) regions are included in the empirical analysis. Eventually, due to limited data availability, most NUTS 
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2 regions from Germany, Greece, France, Poland and the UK are not included in the analysis. The data come 
from the Eurostat database. 


Table 1: Correlation coefficients among the inputs and outputs 



HIA 

HBA 

GDP 

POPDEN 

SECED 

TERED 

EDTOT 

TERAT 

EMPLOY 

Inputs (discretionary) 

HIA 

1.000 


HBA 

0.456 

1.000 


Inputs (non-discretionary) 

GDP 

0.375 

0.138 

1.000 


POPDEN 

-0.054 

0.104 

0.039 

1.000 


Outputs 

SECED 

0.235 

0.062 

0.161 

-0.098 

1.000 


TERED 

0.264 

0.198 

0.228 

-0.086 

0.102 

1.000 


EDTOT 

0.519 

0.104 

0.141 

0.016 

0.424 

0.595 

1.000 


TERAT 

0.473 

0.458 

0.248 

-0.007 

0.162 

0.600 

0.569 

1.000 


EMPLOY 

0.577 

0.308 

0.252 

-0.053 

0.083 

0.205 

0.263 

0.390 

1.000 


Source: Eurostat, 2013; own calculations. 


The degree of correlation between inputs and outputs is an important issue that has a great impact on the 
robustness of the DEA model. Thus, a correlational analysis is crucial to establish appropriate inputs and 
outputs. On one hand, if very high correlations (higher than 0.95) are found between an input variable and any 
other input variable (or between an output variable and any of the other output variables), this input or output 
variable may be thought of as a proxy for the other variables. On the other hand, if an input variable has a very 
low correlation with all the output variables (or an output variable has a very low correlation with all the input 
variables) this may indicate that this variable does not fit the model. In our correlation analysis we could not find 
any evidence of a very high correlation between the input variables (nor between the output variables) (see 
Table 1). Accordingly, this is a reasonable validation of the presented DEA models. Different inputs and 
outputs/outcomes are tested in three models (see Table 2). The program used for calculating the relative 
efficiency scores is the Frontier Analyst 4.0 software. 


Table 2: Input and output/outcome set for the DEA 


Model 

Inputs 

Outputs/Outcomes 

I 

o Households that have Internet access 
at home (% of households) 

(HIA) 

o Pupils and students in upper secondary and post¬ 
secondary non-tertiary education (ISCED 3-4) % of 
the population aged 15-24 years (SECED) 
o Students in tertiary education (ISCED 5-6) (% of 
the population aged 20-24 years) (TERED) 

II 

o Households that have Internet access 
at home (% of households) 

(HIA) 

o Percentage of households with 
broadband access in relation to 
households with Internet access 
(% of households) (HBA) 

o Pupils and students at all levels of education 
(ISCED 0-6) (% of total population) (EDTOT) 
o Tertiary educational attainment (age group 25-64, 
in %) (TERAT) 

III 

o Households that have Internet access 
at home (% of households) 

(HIA) 

o Percentage of households with 
broadband access in relation to 
households with Internet access (% of 
households) (HBA) 

(Non-discretionary) 

o Regional gross domestic product (PPS 
per inhabitant in % of the EU-27 
average) (GDP) 

o Population density (Inhabitants per 
km 2 ) (POPDEN) 

o Pupils and students at all levels of education 
(ISCED 0-6) (% of total population) (EDTOT) 
o Employment rate (age group 15-64, in %) 
(EMPLOY) 


Source: Eurostat, 2013; own calculations. 
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In addition, to evaluate the impact of selected information society indicators on education we calculate partial 
correlation coefficients for different information society (independent) and education (dependent) variables (see 
Table 3). To see whether, for instance, Internet accessibility has any impact on educational outputs and 
outcomes, we calculate the partial correlations between different variables while controlling for the other 
variable(s). We identify that all selected educational output and outcome variables show a moderate and positive 
(statistically significant) correlation with the share of households that have Internet access at home (HIA) when 
controlling for the percentage of households with broadband access in relation to households with Internet access 
( HBA ). Indeed, the impact of the share of households with Internet access at home is moderate and positive as 
the partial coefficient ranges from 0.198 (with TERED) to 0.413 (with EDTOT). The important information 
society variables that also influence the selected outputs/outcomes are the share of pupils and students at all 
levels of education ( EDTOT) and the share of tertiary educational attainment ( TERAT) as the partial coefficient 
reached 0.214 and 0.309, respectively. Nevertheless, the single most important related variable is the share of 
households that have Internet access at home (HIA). Therefore, Internet access seems to be a crucial infonnation 
society variable in order to improve education, training and employment at the regional level within the EU-27. 
However, in order to achieve these educational and employment improvements, it will be necessary to maintain 
existing efforts to provide affordable access to the Internet via broadband and to educate people with the 
necessary skills to enable them to access and exploit the riches of the Internet also for educational and training 
purposes. 


Table 3: Partial correlation coefficients (n=146) 


Output/outcome variables 

Input variables 


Households that 

have Internet access 
at home (% of 
households) (HIA) 

Percentage of households 
with broadband access in 
relation to households with 
Internet access (% of 
households) (HBA) 

Pupils and students in upper secondary and post-secondary 
non-tertiary education (ISCED 3-4) % of the population 
aged 15-24 years (SECED) 

0.233*** 

0.053 

Students in tertiary education (ISCED 5-6) (% of the 
population aged 20-24 years) (TERED) 

0.198** 

0.091 

Pupils and students in all levels of education (ISCED 0-6) 
(% of total population) (EDTOT) 

0.413*** 

0.214** 

Tertiary educational attainment (age group 25-64, in %) 
(TERAT) 

0.335*** 

0.309*** 

Employment rate (age group 15-64, in %) (EMPLOY) 

0.515*** 

0.063 


Source: Eurostat, 2013; own calculations. 


EMPIRICAL RESULTS 

The results of the output-oriented VRS formulation of the DEA analysis (based on Models I—III in Table 2) 
suggest a relatively high level of inefficiency in transforming the benefits of the information society into 
educational outputs/outcomes in selected EU (NUTS 2) regions and, correspondingly, that there is significant 
room to improve educational outputs and outcomes. Indeed, the empirical results show that the total number of 
efficient regions varies significantly from one model to another. There are only six technically efficient EU 
regions in Model I (see Table 4). However, with only 21.0% of households that have Internet access at home 
Severozapaden (BG) (which is also one of the least developed regions within the EU) has the lowest level of 
Internet accessibility among all regions in the sample. Consequently, only Reg. Bruxelles/Brussels Gewest (BE) 
and Prov. West-Vlaanderen (BE) can serve as a good benchmark for the other regions as they both have above- 
average input (% of households that have Internet access). The least efficient regions come from cohesion 
countries, such as Spain and Slovakia, mainly as a result of their relatively low educational outputs, for instance, 
students in tertiary education, ranging from 34.2% (Illes Balears (ES)) to 77.7% (Cataluna (ES)) (for instance, 
the regional sample group average is around 60%). In order to enhance the reliability of the findings, additional 
inputs and outputs/outcomes were introduced, resulting in models II and III (for details also see Table 2). 
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Table 4: Relative Efficiency (Model I) 


121 regions 

The most efficient regions 

The least efficient regions 

Bucuresti - Ilfov (RO) 

100.0 

Illes Balears (ES) 

28.3 

Praha (CZ) 

100.0 

Cataluna (ES) 

38.6 

Prov. West-Vlaanderen (BE) 

100.0 

Ciudad Autonoma de Melilla (ES) 

40.6 

Reg. Bruxelles / Brussels 

Gewest (BE) 

100.0 

Canarias (ES) 

41.5 

Severen tsentralen (BG) 

100.0 

Aragon (ES) 

42.2 

Severozapaden (BG) 

100.0 

Comunidad Foral de Navarra (ES) 

42.8 

Prov. Oost-Vlaanderen (BE) 

98.6 

Zapadne Slovensko (SK) 

43.0 

Prov. Limburg (BE) 

98.3 

Ciudad Autonoma de Ceuta (ES) 

43.1 

Bratislavsky kraj (SK) 

95.5 

Cantabria (ES) 

43.9 

Yuzhen tsentralen (BG) 

94.8 

Vychodne Slovensko (SK) 

44.3 


Average Efficiency Score 

65.5 

Standard Deviation 

17.8 

No. (%) of Efficient Regions 

6 (5.0%) 


Note: The regions in bold have an above-average number of households that have Internet access. 

Source: Eurostat, 2013; own calculations 

Adding another input and two different outputs in the form (Model II) of the percentage of households with 
broadband access in relation to households with Internet access (% of households) (H1A) and pupils and students 
at all levels of education (% of total population) (EDTOT)/tertiary educational attainment (age group 25-64, in 
%) ( TERAT ), respectively, the results again show that those regions from Belgium, i.e. Reg. Bruxelles/Brussels 
Gewest and Prov. Brabant Wallon seem to be the technically most efficient EU regions (see Table 5). Not 
surprisingly, increasing the number of outputs/outcomes in a relatively small sample leads to a higher number of 
efficient regions. In general, the rankings among efficient regions remain relatively stable in comparison to 
Model I (with some new efficient regions from Bulgaria, Romania, Spain and Italy, primary as a result of the 
low levels of additional input (% of households with broadband access) included in the model). 


Table 5: Relative Efficiency (Model II) 


133 regions 

The most efficient regions 

The least efficient regions 

Bucuresti - Ilfov (RO) 

100.0 

Burgenland (AT) 

47.3 

Molise (IT) 

100.0 

Niederosterreich (AT) 

51.2 

Nord-Est (RO) 

100.0 

Stredni Cechy (CZ) 

53.1 

Pais Vasco (ES) 

100.0 

Valle d'Aosta/Vallee d'Aoste (IT) 

54.4 

Prov. Brabant Wallon (BE) 

100.0 

Lombardia (IT) 

54.6 

Reg. Bruxelles / Brussels 

Gewest (BE) 

100.0 

Karnten (AT) 

55.9 

Severen tsentralen (BG) 

100.0 

Piemonte (IT) 

57.3 

Severozapaden (BG) 

100.0 

Kozep-Dunantul (HU) 

57.7 

Sud-Vest Oltenia (RO) 

100.0 

Illes Balears (ES) 

57.8 

Yugozapaden (BG) 

100.0 

Sredisnja i Istocna Hrvatska (HR) 

58.0 


Average Efficiency Score 

76.9 

Standard Deviation 

12.8 

No. (%) of Efficient Regions 

10(7.5%) 


Note: The regions in bold have above-average number of households that have Internet access. 
Source: Eurostat, 2013; own calculations 


However, among the least efficient regions, surprisingly, some regions from Austria are included (such as 
Burgenland and Niederosterreich). We can find the main reason for this relative inefficiency in the 
comparatively highly developed information society and well-below-average educational outputs/outcomes in 
both mentioned regions (for instance, in Burgenland and Niederosterreich they have only 15.0% and 17.6% of 
tertiary educational attainment, respectively). 


Copyright © The Turkish Online Journal of Educational Technology 


58 













































T 


TOJET: The Turkish Online Journal of Educational Technology - April 2014, volume 13 issue 2 



Model III includes two additional (non-discretionary) inputs (GDP and POPDEN) and one additional 
output/outcome variable to the EDTOT variable, i.e. employment rate (age group 15-64, in %) {EMPLOY). 
According to this model, even 31 regions are efficient (see Table 6). The average output efficiency score is 
relatively high at 94.9, meaning that with the same inputs the average region is producing about 5 percent less 
than it should if it were efficient. The worst efficiency performers are regions from Italy (e.g. Sardegna, Puglia 
and Basilicata), Spain (e.g. Principado de Asturias, Cantabria and Canarias) and Croatia (Jadranska Hrvatska and 
Sredisnja i Istocna Hrvatska), all having an underdeveloped information society and also appearing to be highly 
inefficient as they have educational results that are 15-20% lower than those under efficient conditions. 
However, development of the information society, together with its improved efficiency (in terms of educational 
results), which could significantly contribute to a country’s stronger development and growth should remain a 
top priority in the near future for most EU regions, particularly for catching-up regions. 


Table 6: Relative Efficiency (Model III) 


126 regions 

The most efficient regions 

The least efficient regions 

Aland (SE) 

100.0 

Sardegna (IT) 

77.8 

Bratislavsky kraj (SK) 

100.0 

Principado de Asturias (ES) 

79.2 

Bucuresti - Ilfov (RO) 

100.0 

Jadranska Hrvatska (HR) 

81.5 

Centro (PT) 

100.0 

Puglia (IT) 

81.7 

Ciudad Autonoma de Ceuta (ES) 

100.0 

Sredisnja i Istocna Hrvatska(HR) 

82.3 

Ciudad Autonoma de Melilla 
(ES) 

100.0 

Basilicata (IT) 

83.1 

Del-Alfold (HU) 

100.0 

Sicilia (IT) 

84.4 

Flevoland (NL) 

100.0 

Cantabria (ES) 

84.5 

Midtjylland (DK) 

100.0 

Canarias (ES) 

85.1 

Molise (IT) 

100.0 

Abruzzo (IT) 

85.7 

Moravskoslezsko (SK) 

100.0 

Illes Balears (ES) 

86.9 

Niederiisterreich (AT) 

100.0 

Kozep-Magyarorszag (HU) 

87.3 


Average Efficiency Score 

94.9 

Standard Deviation 

5.2 

No. (%) of Efficient Regions 

31 (24.6%) 


Note: The regions in bold have above-average number of households that have Internet access. 
Source: Eurostat, 2013; own calculations 


On the other hand, regions from Belgium (e.g. Reg. Bruxelles/Brussels Gewest, Prov. Oost-Vlaanderen), 
Netherlands (e.g. Flevoland, Utrecht), Sweden (e.g. Aland, Stockholm, Smaland med oama), Denmark (e.g. 
Midtjylland) and even Slovakia (Bratislavsky kraj) are among the most efficient performers as they all are 
among the best output/outcome educational performers with a relatively high share of households that have 
Internet access at home and also broadband access. Therefore, these regions could all serve as a good benchmark 
for the other EU regions in terms of their information society development and their efficiency. 

CONCLUSION 

This paper joins the efforts of other scholars in investigating information society efficiency by applying a non- 
parametric methodology at the regional level in the EU. In this respect, the Data Envelopment Analysis (DEA) 
technique was presented and then applied to a wide range of EU-27 (NUTS 2) regions to evaluate the technical 
efficiency of harnessing information society riches also for educational and training purposes. The research 
findings suggest that Reg. Bruxelles/Brussels Gewest, Prov. Oost-Vlaanderen (Belgium), Flevoland, Utrecht 
(Netherlands), Aland, Stockholm, Smaland med oama (Sweden), Midtjylland (Denmark) and Bratislavsky kraj 
(Slovakia) belong to the best performing NUTS 2 regions located on the regional efficiency frontier. These EU 
regions could also serve as peers to improve the efficiency of the less efficient ones. The results confirm the idea 
that regions with a mature information society generally enjoy better educational outputs and results compared to 
regions still developing their information society pattern. In contrast, a wide range of NUTS 2 regions from 
Eastern and Southern Europe is characterized by an extremely low rate of information society development 
(most of the regions in Bulgaria and Romania) and efficiency in terms of educational outputs/results, particularly 
in Spain (e.g. Illes Balears, Canarias), Italy (e.g. Sardegna, Sicilia), Czech Republic (e.g. Stredni Cechy) and 
Hungary (e.g. Kozep-Dunantiil), suggesting there is still significant potential to develop the information society 
and improve educational results in many EU regions, particularly those from catching-up EU member states. 
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Nevertheless, a few limitations of the presented empirical study should be pointed out. Firstly, the application of 
the presented techniques is hampered by a lack of suitable data. Quality data are called for because the 
techniques available to measure efficiency are sensitive to outliers and may be influenced by exogenous factors. 
Secondly, the precise definition of inputs, outputs and outcomes may significantly influence the results. Finally, 
it seems important to bear in mind that by using a non-parametric approach, and in spite of DEA being an 
established and valid methodology, differences across countries are not statistically assessed and this may be 
considered a limitation of the methodology. Flence, further research is clearly needed to eliminate the above 
deficiencies, in particular to test the influence of environmental factors on ICT efficiency. 
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